Enabling and Analyzing Natural Interaction with Functional Virtual Prototypes

Functional validations of virtual prototypes are a promising application area of immersive Virtual Reality (Moehring & Froehlich, 2011). Through the interactive simulation of operating procedures such analyses aim at providing insight about virtual prototypes, e. g. concerning visibility aspects, reachability of control elements, and ergonomics. A key requirement is the enabling of natural interaction defined by Zachmann as "interaction which imitates that same interaction in the real world as close as possible" (Zachmann & Rettig, 2001). In immersive VR, natural interaction can be realized through data gloves and motion tracking devices. However, a challenging task remains in the modeling and simulation of interactions with dynamic control elements such as sliders, switches etc. (Moehring & Fröhlich, 2010). We have devised and implemented a framework that not only aims at simplifying the specification of such control actuators for interactive virtual environments but also facilitates the recording, automated classification, and analysis of natural interactions with such control actuators. The presented approach builds on the European Standard EN 894-3 (DIN894-3, 2006) and covers all types of control actuators defined therein. This standard defines systematic guidelines for the choice and configuration of control actuators to ergonomically design machinery. A characterizing feature of control actuators in the real world is given by their function: Controlling the behavior of machines and other appliances. In order to model the triggering of changes of environment objects in the simulation, our framework allows to associate control actuators with interaction events. Interaction events may be fired continuously, e. g. during movement of sliders, or when discrete states are reached, e. g. switches. Besides making interactive simulations more realistic, interaction events can also be recorded to allow later playback and analysis of the interaction. For this, detailed timing information is stored in the interaction events. For example, interaction events may be used for the animation of virtual humans that imitate the user interactions when operating virtual prototypes. Contributions of our research include an XML-based modeling language for control actuators that can be seen as both an abstraction layer and an extension to lower-level rigid body physics modeling capabilities provided e. g. by Collada (Khronos Group, 2008) and X3D (International Organization for Standardization, 2008). Further, we produce a reference implementation of control actuators with direct object manipulation, an interaction database for analyzing interaction events including recognizing basic interactions and grasp classification, and a powerful graphical analysis tool. Through this, the process of adding interactive elements to virtual prototypes can be drastically simplified. 11